Comparing Neural Network, Logistic Regression, and Discriminant Analysis for Knowledge Representation and Classification Explanation

Many research studies have proved neural networks as a viable alternative to statistical models for classification tasks. However, compared with statistical models, neural networks have had the drawback of being unable to explain its classification logic until the development of rule extraction algorithms from trained neural networks. This research attempts to compare the results of the rule extraction algorithm GLARE with logistic regression and discriminant analysis in terms of their ability to identify important predictor variables and handle different levels of difficult classification tasks. Our experimental results show that GLARE can precisely identify important predictor variables as its statistical counterparts. In addition, GLARE's extracted rules generate higher correct classification rates than statistical models in a moderately difficult classification task. Most importantly, GLARE can reveal nonlinear relationship between predictor variables and tobe-predicted classes, which many statistical classifiers cannot.